DOCUMENT RESUME 



ED 460 536 



EC 306 912 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



AVAILABLE FROM 

PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Garland, Ann F. ; Litrownik, Alan J. ; Mitchell, Barbara E.; 
Levine, Zachary A. 

The Development of a Standardized Assessment Protocol in a 
Community Mental Health Treatment Program. 

1996-02-00 

8p.; In: A System of Care for Children's Mental Health: 
Expanding the Research Base. Proceedings of the Annual 
Research Conference (9th, Tampa, FL, February 26-28, 1996); 
see EC 306 844. Table is not available from ERIC. 

For full text: 

http : //rtckids . fmhi .usf . edu/Proceed9 thZ9thprocindex.htm. 
Reports - Research (143) -- Speeches/Meeting Papers (150) 

MFOl/PCOl Plus Postage. 

Adolescents; *Child Abuse; Children; *Community Programs; 
Data Collection; *Emotional Disturbances; Evaluation 
Methods; *Mental Health Programs; *Outcomes of Treatment; 
Program Effectiveness; Recordkeeping; Sexual Abuse 



ABSTRACT 



This paper summarizes the process of instituting a 
standardized mental health assessment protocol in a community health 
treatment program serving maltreated youth. The assessment program was 
intended to: (1) provide descriptive data regarding the populating utilizing 
the service; (2) examine the efficacy of therapeutic interventions and 
.monitor program quality; and (3) supply individual clinical data to mental 
health providers for use in treatment planning and monitoring. Data for 242 
subjects receiving outpatient services for sexual, physical, or emotional 
abuse and neglect have been collected. Analysis is in terms of the three 
program goals. Concerning the goal of providing descriptive data, although 
assessments were conducted on only 60 percent of patients, data have been 
collected on demographics, maltreatment history, mental health service use 
history, and treatment expectancies, and mental health and family problems 
and competencies. Concerning use for assessing therapeutic efficacy, 
follow-up data have been collected for only 59 youth and their families. 
Concerning supplying individual clinical data to mental health providers, 
clinicians have reported satisfaction with assessment summaries. (Contains 16 
references.) (DB) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 
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Although there are many research studies evaluating the 
efficacy of various mental health interventions for children 
and families, there are very few reports of ongoing 
standardized evaluation of treatment outcomes in 
community mental health services (Weisz & Weiss, 1989; 
Weisz, Weiss, & Donenberg, 1992). It is becoming 
increasingly important that outcome assessment be both 
widely applied and linked to data on patient characteristics 
and service utilization (Sederer, Hermann, & Dickey, 1995). 
Community mental health services are experiencing 
increased internal and external pressure to document the 
efficacy of their interventions and the effectiveness of the 
service system. These pressures have intensified in this era 
of managed care and diminished resources for mental health 
services. Furthermore, these pressures dictate that 
procedures be utilized which are both time and cost efficient 
(Kurkland, 1995). Unfortunately, increased pressure to 
evaluate outcomes has not necessarily coincided with the 
development of well established guidelines for 
implementing outcomes evaluation. Service settings are left 
with the difficult task of designing their own protocols, with 
or without consultation from researchers who have the 
necessary expertise in measurement and analysis. 



The purpose of this study was to describe the process of 
instituting a standardized mental health assessment protocol 
in a community mental health treatment program serving 
maltreated youth. The protocol was specifically not 
implemented as a research project, but rather as a low cost 
component of standard clinical practice at the clinic. The 
goals of this assessment program were to: 

• Provide descriptive data regarding the population 
utilizing the service, and describe individual and 
group differences in this population 

• Examine the efficacy of therapeutic interventions and 
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provide a mechanism for monitoring quality 
assurance 

• Supply individual clinical data to mental health 
providers for use in treatment planning and 
monitoring. 

Method 

Service Site and Sample 

Children's Hospital Center for Child Protection (CCP) 
Mental Health Treatment Program 

The CCP mental health treatment program is an outpatient 
therapy program specifically for children and adolescents 
who have experienced maltreatment in the form of sexual, 
physical, or emotional abuse, or neglect. The majority of 
patients are referred for sexual abuse. Individual, group, and 
family treatment are available from an interdisciplinary 
team of psychologists, social workers, counselors, and 
trainees. In the fiscal year 1994, the program served more 
than 250 children and their parents. A standardized 
assessment protocol was introduced in mid 1 994 and data 
for 242 subjects have been collected. 

Description of Sample 

Sixty-two percent of the subjects are female; the mean age 
was 8.7 years (SD = 4.0; range = 2-19). Approximately half 
of the sample was Caucasian (47%); African Americans and 
Hispanics composed 20% each of the sample, with the 
remainder classified as "Other." The majority of the families 
(61%) had total gross incomes of less than $20,000. 
Approximately two thirds of the children were referred 
primarily because of sexual abuse. Six-month follow-up 
data has been more difficult to obtain, but is currently 
available for 59 families. 

Measures 

Table 1 . below, presents the measures selected for the 
assessment program, by domain of assessment and 
informant. 

Issues which were addressed in the selection of instruments 
included: 

• Goals and objectives of intervention (i.e. relevance of 
measure) 

• Psychometric properties of instruments for the 
population being served (e.g., SES, race/ethnicity, 
culture, age, etc.), and 

• Pragmatic issues, such as time of administration, 
costs, personnel, and training required. 

Procedure 

Assessment measures were administered by graduate 



3 



Assessment measures were administered by graduate 
students and research assistants who received training in the 
administration of the standardized instruments and some 
limited clinical interviewing skills. Families were scheduled 
for an intake assessment prior to initiating treatment. Data 
regarding the assessment were used to assign families to 
specific types of treatment and providers. A summary of the 
assessment data was provided to the clinician prior to the 
family's first visit. Follow-up interviews were scheduled six 
months after the intake assessment and the same battery of 
instruments was repeated. 

Results 

Results are presented and discussed based on the relative 
success of achieving the program goals and the obstacles 
confronted within the process. 

Goal #1: Provide descriptive data regarding the 
population utilizing the services. 

A great deal of descriptive data has been collected on the 
242 youth assessed upon entering the mental health 
treatment program. Such data includes demographics, 
maltreatment history, mental health service use history, 
treatment expectancies, and standardized data regarding 
mental health and family problems and competencies. 

There are, however, limitations in the usefulness of these 
data; variability in the clinical staffs comfort and familiarity 
with the instruments jeopardizes the reliability of results. 
Also, the data may not be representative of all youth and 
their families utilizing the service due to the difficulty 
involved in successfully assessing all children and their 
parents prior to treatment (Boren et al., 1996). Only 
approximately 60% of those who began treatment were 
assessed. Anecdotal data suggests that the reasons for 
missing the intake assessment include, crisis intervention 
implemented prior to assessment, scheduling difficulties, 
and language differences. 

Goal # 2 : Examine the efficacy of the therapeutic 
interventions and provide a mechanism for quality 
assurance 

The utility of the assessment program for evaluating 
outcomes obviously relies on the ability to collect follow-up 
data, as well as on the representative coverage of the 
assessments, as discussed above. To date, follow-up data 
have been collected for only 59 youth and their families, too 
few to provide a meaningful assessment of treatment 
outcomes. Uncertainty regarding whether a true 
cross-section of the patients is being considered in outcome 
evaluation may arise due to a lack of full participation in 
follow-up interviews. This is largely the result of difficulty 
in tracking some families after their initial interview. 

A number of additional obstacles to obtaining reliable 
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A number of additional obstacles to obtaining reliable 
follow-up data are worth addressing. One such obstacle, 
related to clinicians' variable familiarity with the 
instruments, is the potential for variable commitment of the 
treatment program staff to standardized assessment. The 
requirements, for example, of standardization may run 
counter to the intentions of many clinical providers to adapt 
to the diverse, idiosyncratic, and changing needs of 
consumers, thereby seeming incongruent or even 
contradictory to the clinical and/or administrative 
procedures of a mental health treatment program. Indeed, 
lack of consistent commitment could impact the scheduling 
of intake interviews and follow-ups, further compromising 
the universality of outcome data collection. 

Due to their integral role in the success of such an 
assessment protocol, clinicians must be highly involved in a 
collaborative implementation of it. The researchers 
introducing the procedure must emphasize their role as 
consultants, and must transfer administrative functions to 
the clinic in a timely manner. Training and data analysis can 
remain the researchers' responsibility; however, it is 
imperative to involve clinicians early in order to establish it 
as a standard clinical practice, rather than a research project. 
This entails getting input from clinical staff regarding 
research questions and constructs to measure prior to the 
protocol's implementation. 

This assessment program was intentionally implemented as 
an integral component of ongoing clinical practice, as 
opposed to a discrete research project. Thus, there was no 
attempt to implement an experimental research design with 
random assignment to treatment conditions, etc. While this 
approach offered clinical advantages, there are limitations in 
the interpretations of results due to the lack of standardized 
conditions. 

Goal #3: Supply individual clinical data to mental health 
providers for use in treatment planning and monitoring 

Clinicians have reported satisfaction with the assessment 
summaries, although the treatment program manager reports 
that the summaries are not frequently used for assignment to 
treatment. Thus, possible variability in clinical utility is an 
issue of which to be aware. 

Changes in the data reporting methods have been instituted 
due primarily to reimbursement issues. Reimbursement 
issues arose because the assessment protocol did not match 
the categories allowed for reimbursable expenses in many 
public and privately funded managed care programs. The 
procedures were therefore changed, and made more costly, 
so that the requirements for reimbursement could be met. 
Instead of an assessment summary completed by a 
paraprofessional, a full psychological evaluation report was 
produced by a licensed clinical psychologist, and the 
assessment was billed as psychological testing. This change, 
in some respects, was contrary to the initial purpose of the 
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in some respects, was contrary to the initial purpose of the 
protocol, which was to provide a low cost assessment 
program in the implementation of such a protocol. 

Discussion 

It is feasible to implement a standardized assessment 
protocol at relatively low cost in a community mental health 
treatment program. There are, however, several obstacles 
that need to be addressed including, lack of full coverage, 
reimbursement requirements, and tracking of families for 
follow-up. The introduction of standardized assessment to a 
treatment program requires the following: 

• Early involvement and training of staff; 

• Planning for fiscal impact and flexibility in 
reimbursement plan; 

• The selection of feasible measures (e.g., time, cost, 
and training efficient), the data from which will meet 
the goals of the intervention; 

• Well developed database and client tracking systems; 

• Linkage of assessment data to service use data; and 

• Timeline of implementation and plans for transition 
from trial to standard clinical practice. 

With these issues addressed, a successful standardized 
assessment protocol can be implemented with benefits to 
clinicians, administrators, and researchers alike. 
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